The Probabilistic Hitting Set Paradigm: a General Framework for Search and Detection in Dynamic Social Networks

We formulate and study the Probabilistic Hitting Set Paradigm (PHSP), a general framework for design and analysis of search and detection algorithms in large scale dynamic networks. The PHSP captures applications ranging from monitoring new contents on the web, blogosphere, and Twitterverse, to analyzing influence properties in social networks, and detecting failure propagation on large electronic circuits. The Probabilistic Hitting Set Paradigm (PHSP) defines an infinite time generating process that places new items in subsets of nodes, according to an unknown probability distribution that may change in time. The freshness or relevance of the items decay exponentially in time, and the goal is to compute a dynamic probing schedule that probes one or a few nodes per step and maximizes the expected sum of the relevance of the items that are discovered at each step. We develop an efficient sampling method for estimating the network parameters and an efficient optimization algorithm for obtaining an optimal probing schedule. We also present a scalable solution on the MapReduce platform. Finally we apply our method to real social networks, demonstrating the practicality and optimality of our solution.